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SYSTEM AND METHOD FOR 
DYNAMIC DISTRIBUTED COMMUNICATION 



TECHNICAL FIELD 

The present invention relates to dynamically modifying 
5 communication network components and communication paths. 

BACKGROUND ART 

Communication systems are increasingly required to provide a wide 
range of services, including different forms of information communicated with 
different communication characteristics. Information forms include voice, data, 

10 video, telemetry, and the like. Communication characteristics include quality, 
complexity, latency, reliability, cost, availability, portability, and the like. 
Infrastructure such as telecommunication systems, the Internet, and cable systems 
exist to provide long-haul routing and information content sourcing. However, 
difficulty remains in delivering this information to customers. This is particularly 

15 the case if the customer is located in a rural location, is communicating through 
portable equipment, or is mobile. 

Traditionally, communication service providers have relied on copper 
wire or coaxial cable to connect distribution sites and subscriber premises. 
However, increases in the number of users, number and type of communication 

20 devices per user, and the information rate per device has strained the ability for 
traditional communication systems to provide the necessary bandwidth and 
flexibility. Various technologies including digital subscriber line (DSL) and video 
modems offer broadband access to the subscriber over existing copper or coaxial 
loop technologies. Fiber-to-the-home offers broadband access through additional 

25 wireline connections. While each technology has broadband delivery properties, 
each is subject to physical and signaling limitations that restrict availability in certain 
locations and for certain applications. For example, VDSL is limited to within 
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approximately one kilometer of a connecting central office. Further, each central 
office usually cannot support high-speed access for every customer within the central 
office coverage area. Similarly, service provided by two-way cable modems is 
limited to the installation of coax, hybrid fiber coax and fiber-based systems by the 
5 cable provider as well as capabilities of the supporting central office. Currently, 
many premises do not have access to high-speed wireline voice, data, and video 
services due, in part, to the expense and difficulty in constructing or expanding 
central offices. 

Wireless systems, such as PCS and cellular systems, offer an 
10 alternative to wireline services. Typically, wireless systems include a centralized 
mobile switching center (MSC) responsible for routing calls, tracking user locations, 
billing information, and connectivity with other communication systems. The MSC 
is connected to base station controllers (BSCs), each of which supports one or more 
base transceiver stations (BTSs). Each BTS supports one or more cells based on the 
15 number and configuration of antennas supported by the BTS. Customers 
communicate with the wireless system through radio telephones when the telephone 
is within the coverage range of a cell. When a call is placed, a circuit-switched 
connection is established from the telephone, through the BTS and BSC, to the MSC. 
The MSC determines the destination and, if the destination is to another telephone 
20 within the wireless system, establishes a circuit-switched connection to the 
destination telephone. If the destination is outside of the wireless system, the MSC 
routes the call to a service provider for the outside destination. 

Cellular and PCS wireless systems have traditionally focused on 
voice. However, recent introduction of digital radio in both the 800 MHz and 1.9 
25 GHz ranges permits low-speed data services, such as text messaging and one-way 
paging, to be implemented. Also, wireless modems implementing wireless 
application protocols may be incorporated into hand-held devices such as lap top 
personal computers and personal digital assistants (PDAs). Such devices may offer 
access to the Internet at rates up to 19.2 kilobits per second. 
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Several problems have developed with traditional wireless systems. 
First, because all calls are processed by the central MSC, it can become a bottleneck 
limiting the number of users supported- Second, if the MSC fails, the system 
becomes inoperative. Third, the BSC and BTS architecture is fairly rigid, requiring 
5 manual reconfiguration in the MSC when components are added or deleted. Fourth, 
circuit-switched channel connections establish an upper bandwidth, limiting 
communication rates. Fifth, bandwidth is wasted if the communication rate does not 
reach the maximum established by the communication channel. Sixth, high-speed 
data and video applications are impractical on wireless telecommunication systems 
10 designed primarily for voice. 

♦ An alternative wireless communication system is the multi-channel 
multipoint distribution system (MMDS). In MMDS, an antenna broadcasts high- 
bandwidth content to local receiving antennas. In the case of Internet access, the 
return path is typically through slow telephone lines. MMDS, like wireline cable 

15 systems, is asymmetrical, making it best suited for distribution of one-way 
entertainment signals such as television. MMDS also typically operates at a higher 
carrier frequency than PCS systems, requiring receiving antennas to be positioned 
in line-of-site with the transmitting antenna. MMDS is typically deployed in a single 
cell broadcast architecture. Other wireless communication systems, such as local 

20 multipoint distribution system (LMDS) may be deployed in a cellular fashion. 
However, because these systems typically operate at a high carrier frequency, 
receiving antennas must be positioned in line-of-sight with the transmitting antenna. 

What is needed is a communication system that provides voice, data, 
and video signals at rates appropriate for each user. The system should make 

25 efficient use of bandwidth, allocating only the bandwidth necessary for a particular 
communication. The system should be flexible, permitting automatic addition and 
deletion of network components. The system should have distributed routing and 
service provisioning to prevent bottlenecks, permit scaling, and increase reliability 
and robustness. The system should support wireless subscriber units, 

30 accommodating a wide variety of fixed, portable, and mobile user communication 
devices. The system should support high-speed symmetric and asymmetric 
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communication for applications such as video conferencing, real-time distributed 
document sharing, and the like. The system should also easily and rapidly adapt to 
mobile and portable users that change location throughout the communication 
system. 

5 DISCLOSURE OF INVENTION 

The present invention provides automatic and dynamic addition of 
network routing or distribution elements to a distributed communication network. 
This permits rapid, scalable, and inexpensive growth of the communication network, 
allowing for a flexible system that can be easily modified to meet customer needs, 

10 balance resources, and satisfy physical constraints and government regulations. The 
present invention also provides automatic and flexible routing of information packet 
through a distributed communication system. This permits users to enter the 
communication system at any point and to freely move about within the 
communication system without having to register or otherwise manually notify the 

15 communication system. 

A new routing element may be dynamically added to a network of 
routing elements by first establishing a connection between the new routing element 
and an existing routing element in the network. The connection may be either 
wireless or wireline. At least one address is assigned to the new routing element. 
20 Each assigned address comes from a pool of addresses maintained at the existing 
routing element. At least one pool of addresses is issued to the new routing element, 
permitting the new routing element to dynamically add yet another new routing 
element to the network of routing elements. 

The assigned address may be one or more from a set including an 
25 ATM address and an IP address. Likewise, address pools may be pools of ATM 
addresses and pools of IP addresses. Other types of addresses may be included or 
substituted, such as TCP, 802.x, and the like. Any assignable address that can 
distinguish between network elements may be used. 
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In one embodiment, a supervisor supplies pools of ATM end user 
addresses and pools of IP addresses. Each distribution point obtains a pool of ATM 
end user addresses and a pool of IP addresses from the supervisor. When a 
distribution point receives a request from a new network element to join the 
5 network, the distribution point provides the new network element with an ATM end 
user address from the pool of ATM end user addresses and an IP address from the 
pool of IP addresses. An access point may also receive address pools and function 
as a distribution point in support of additional access points. 

Information packets are routed through the distributed routing network 
10 by determining a forwarding equivalency class (FEC) for each subscriber unit 
accessing the network. The FEC to which each subscriber unit belongs is based on 
the point at which the subscriber unit accesses the network. The forwarding 
equivalency class for each subscriber unit is updated if the subscriber unit accesses 
the network at a different point. Information packets are routed from a distribution 
15 point by determining the next point connected to the distribution point based on the 
FEC for the destination subscriber unit specified in the packet. Packets are 
preferably IP packets, but any form of packetized information may be routed in this 
manner. 

Routing elements may be of a variety of types. Any distribution 
20 element that receives or forwards information packets may be used. For example, 
routing elements may include distribution points and access points in a 
communication network. In one aspect of the present invention, distribution points 
form peer-to-peer relationships with each other. Access points form client-server 
relationships with distribution points. If multiple addresses are assigned to each 
25 routing element, relationships between routing elements may be established 
concurrently across more than one layer. For example, IP routing may be 
established at the network layer and ATM switching at the data link layer with 
routing elements communicating in a peer-to-peer or client-server relationship at 
each layer. 
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Updating of forwarding equivalency classes may be accomplished by 
several techniques. An access point determining that a new subscriber unit seeks 
access to the communication system through itself may broadcast this information 
to the distribution points. Similarly, an access point determining that a subscriber 
5 unit is no longer accessing the communication system through itself may broadcast 
this information. The communication system may also include a supervisor in 
contact with the distribution points for determining membership in forwarding 
equivalency classes. 

The above objects and features as well as other objects, features, and 
10 advantages of the present invention are readily apparent from the following detailed 
description of the best mode for carrying out the invention when taken in connection 
with the accompanying drawings. 

BRIEF DESCRIPTION OF DRAWINGS 

FIGURE 1 is a schematic drawing illustrating a portion of a 
15 communication system according to an embodiment of the present invention; 

FIGURE 2 is a block diagram of a distribution point according to an 
embodiment of the present invention; 

FIGURE 3 is a schematic drawing illustrating an implementation of 
a communication system according to an embodiment of the present invention; 

20 FIGURE 4 is a schematic drawing illustrating an initial state for a 

communication system according to an embodiment of the present invention; 

FIGURE 5 is a schematic drawing illustrating the addition of a second 
access point to the communication system of Figure 4 according to an embodiment 
of the present invention; 
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FIGURE 6 is a schematic drawing illustrating a hierarchical routing 
system using ATM/IP switches according to an embodiment of the present invention; 

FIGURE 7 is a schematic drawing illustrating a network of 
distribution points routing packets based on forward equivalency classes; and 

5 FIGURE 8 is a schematic drawing illustrating forward equivalency 

class updating to track a moving subscriber unit. 

BEST MODE FOR CARRYING OUT THE INVENTION 

Referring to Figure 1 , a schematic drawing illustrating a portion of 
a communication system according to an embodiment of the present invention is 

10 shown. A communication system, shown generally by 20, includes a plurality of 
access points 22 which may be, for example, a local radio access point (LRAP). 
Each access point 22 defines coverage area 24 such as, for example, a cell, covering 
a reception range of access point 22. Coverage area 24 may be formed from many 
independent sectors, as may result if access point 22 uses many unidirectional 

15 antennas, or may be a single region resulting from the use of an omnidirectional 
antenna. Subscriber unit 26 within coverage area 24 may establish two-way wireless 
link 28 with access point 22. Subscriber unit 26 may also establish wireline link 29 
with access point 22. Links 28, 29 may be symmetrical or asymmetrical. 
Subscriber unit 26 may be fixed or non-fixed and, if non-fixed, may posses varying 

20 degrees of portability and mobility. Subscriber unit 26 may be a mobile telephone, 
a computer, a video receiver, an audio receiver, a two-way video conferencing 
station, a video game, an information kiosk, a remote sensor, a remote actuator, or 
any other suitable communication device. 

Wireless link 28 may be any form of electromagnetic signaling not 
25 confined to a wire or cable, including energy radiated by antenna as well as visible 
and invisible light. As will be appreciated by one of ordinary skill in the art, 
wireless link 28 may be implemented by any access technology, including CDMA, 
TDM A, FDMA, OFDM, analog, and the like. Modulation techniques that may be 
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used with the present invention include FSK, BPSK, QPSK, m-ary QAM, FM, AM, 
and the like. Further, the invention does not depend on modulation frequency or on 
the use of FDD or TDD. In a preferred embodiment, the access technology, 
frequency, and modulation method for establishing wireless link 28 are based, in 
5 part, on local geography, local regulations, noise and distortion sources, other 
operating wireless systems, cost, or any other suitable parameter. Subscriber unit 
26 and access point 22 may establish wireless link 28 using a plurality of 
combinations of access technology, frequency, and modulation techniques. 

Information transmitted on links 28, 29 may represent voice, data, 
10 video, streaming audio, streaming video, or the like. Types of information include 
speech, facsimile, computer data, entertainment and informational audio and video, 
video game data, telemetry information, security information, and the like. If the 
information occurs as a continuous stream, subscriber unit 26 breaks the information 
into packets prior to packet transmission and reassembles the information stream 
15 from packets after packet reception. Any type of information that exists in packets 
or that may be packetized can be used with the present invention. 

In an embodiment of the present invention, subscriber unit 26 may be 
implemented as part of terminal network controller 30 accepting inputs from and 
providing outputs to information sources including voice equipment 32, computing 
20 equipment 34, telemetry equipment 36, video equipment 38, or any other suitable 
communication equipment. Inputs to terminal network controller 30 may include 
serial data, parallel data, ISDN, standard telephone, xDSL, SR 1394, coaxial cable, 
twisted pair cable, optical fiber, or any other suitable communication protocol, 
method, or medium. 

25 In an embodiment of the present invention, a quality error bit rate is 

established for each subscriber unit 26. This quality error bit rate may be based on 
the location of subscriber unit 26 within communication system 20, the class of 
service assigned to subscriber unit 26, the grade of service assigned to subscriber 
unit 26, the data or transmission rate of service assigned to subscriber unit 26, or 

30 any other suitable parameter. The quality error bit rate may be modified while 
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subscriber unit 26 is within communication system 20 to reflect changing conditions 
such as noise, demand, connectivity, or any other suitable parameter. Applications 
providing services to subscriber unit 26 may adjust these services based on the 
quality error bit rate. For example, an application providing streaming audio and 
5 video may reduce the frame update rate as noise increases, guaranteeing successful 
transmission at a lower information rate. The information rate may be further 
reduced to provide only still pictures and audio if conditions continue to worsen. 
The information rate may also be automatically changed if subscriber unit 26 moves 
between coverage areas 24 with differing transmission capabilities or loads. 

10 Control of the information rate may be achieved by having subscriber 

unit 26 monitor a signaling channel transmitted by access point 22 for each coverage 
area 24. This signaling channel informs subscriber unit 26 when to transmit 
information, how much information to transmit, the information transmission rate, 
and the like. The signaling channel may be controlled by a central supervisor, 

15 described below. 

In an embodiment of the present invention, bandwidth on 
communication link 28 is only consumed when packets containing information are 
transmitted. For example, each subscriber unit 26 surrenders bandwidth on 
communication link 28 when not sending or receiving an information packet. 
20 Packets to be transmitted are queued based on order of arrival, priority, a 
combination of arrival order and priority, or the like. Subscriber unit 26 monitors 
a signaling channel transmitted by access point 22 for each coverage area 24. 
Subscriber unit 26 only consumes bandwidth when instructed to transmit by the 
signaling channel or when receiving packets. 

25 Each access point 22 communicates with at least one distribution point 

40. Distribution point 40 contains both routing and switching functionality. Access 
point 22 may be in contact with one or more radio access distribution points 40 over 
radio link 42, may be wired or cabled to distribution point 40 through wireline link 
44, or may be packaged with distribution point 40. Access point 22 may also be 

30 transformed into distribution point 40, permitting access point 22 to route traffic that 
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neither originated nor terminated with any of its serviced subscriber units 26. 
Distribution point 40 is in communication with at least one additional distribution 
point 40, the collection of interconnected distribution points forming a network of 
distribution points, shown generally by 41. Two distribution points may be 
5 connected by radio link 46 or wireline link 48. 

Distribution points 40 may route packets within distribution point 
network 41 under a variety of protocols such as ATM, TCP/IP, 802.x, or the like. 
In a preferred embodiment, distribution point 40 includes an ATM/IP switch. 
Distribution point 40 then operates at both the IP routing and ATM switching layers 
10 or, in terms of the Open Systems Interconnection (OSI) standard, at both the network 
layer and the data link layer. 

The IP layer operates with a link-state protocol such as the open 
shortest path first (OSPF), quality OSPF (Q-OSPF), or internal gateway routing 
protocol (IGRP) and its derivatives. The IP layer operates as a single autonomous 
system (AS) within the IP frame of reference. Each system 20 will be allocated a 
unique and unambiguous AS number for system management. IP addresses for 
system 20 will use a private IP address space that cannot be routed within public 
systems such as the Internet. Subscriber units 26 within system 20 may be permitted 
access to the private IP address space or may be excluded from the private IP 
address space. When private IP address space is used for subscriber units 26, a 
network address translator (NAT) within system 20 allows subscriber units 26 access 
to the Internet. The ATM layer operates with the private network node interface 
(PNNI) routing protocol. ATM end system addresses (AESAs), managed by the 
service provider for system 20, are used by distribution point network 41. 

The ATM network carries voice traffic and may carry data. Through 
PNNI, the ATM/IP switch participates in switched or signaled virtual connections 
(SVCs). When subscriber unit 26 within system 20 requires voice connectivity, it 
signals such a request, which is serviced by distribution point 40 receiving that 
request. The IP network coresiding with the ATM network is used for delay 
30 insensitive data applications required by subscriber units 26. The IP network is also 
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used for all network management, including management of ATM/IP switches, 
subscriber units 26, gear associated with distribution points 40, and any other 
suitable network component. This includes functions such as alarming, monitoring, 
recovery systems, and the like. While described in the context of a wireless network 
5 application, it is readily apparent that ATM/IP routing as described herein may be 
applied to wireline and mixed wireline-wireless systems as well. 

Each distribution point 40 receives an information packet from either 
another distribution point 40, from subscriber unit 26 in communication with 
distribution point 40 through access point 22, or from an external communication 

10 system. If distribution point 40 determines the information packet is destined for 
subscriber unit 26 within coverage area 24 of access point 22 in communication with 
distribution point 40, distribution point 40 forwards the packet to access point 22 
forming coverage area 24 containing destination subscriber unit 26. If distribution 
point 40 determines the information packet is destined for subscriber unit 26 in 

15 coverage area 24 formed by access point 22 in communication with a different 
distribution point 40, distribution point 40 forwards the packet to one of distribution 
points 40 in communication with distribution point 40. Hence, no central MSC is 
required for routing. Distributed routing removes delays caused by central 
switching, increases the robustness of the communication system 20, increases 

20 network efficiency, and permits simplified expansion or reduction of communication 
system 20 by automatically adding or removing distribution points 40. 

A third option is that distribution point 40 determines that the 
information packet is destined for a destination not part of communication system 20. 
Special distribution points, such as gateway 50, provide a bridge to additional 

25 communication systems 52 including wireless and wireline telecommunication 
systems, video distribution systems, computer network systems such as the Internet, 
packet systems, frame systems, ATM systems, IP systems, private networks, and 
any other suitable communication or information system. If distribution point 40 
determines the information packet is destined for delivery outside of communication 

30 system 20, distribution point 40 forwards the packet to one of distribution points 40 
in communication with gateway 50. 
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In an embodiment of the present invention, communication system 20 
includes communication system interface device 54 operative to format information 
contained in the information packet to pass through telecommunication system 52. 
Communication system interface device 54 may be incorporated into gateway 50 or 

5 may be a separate component of communication system 20. Distribution point 40 
receives at least one information packet from the telecommunication system interface 
device 54 and determines if the at least one information packet destination is to 
subscriber unit 26 within coverage area 24 of access point 22 in communication with 
distribution point 40. Distribution point 40 forwards the at least one information 

10 packet to access point 22 defining coverage area 24 containing subscriber unit 26 if 
the information packet destination is to subscriber unit 26 within coverage area 24 
of access point 22 in communication with distribution point 40 and forwards the at 
least one information packet to one of the additional distribution points 40 in 
communication with distribution point 40 otherwise. 

Each distribution point 40 communicates with supervisor 56. 
Supervisor 56 tracks the locations of subscriber units 26 within communication 
system 20, identifying with which distribution point 40 each subscriber unit 26 is 
currently communicating. Supervisor 56 manages transmission priorities based on 
parameters including load, information type, service requests, location, grade of 
service, information transfer rates, or any other suitable parameter. Supervisor 56 
may also serve as a collection point for alarms and performance measuring of 
communication system 20. Supervisor 56 may further include or interface with 
billing and authentication services. 

In an embodiment of the present invention, supervisor 56 also assigns 
an address to each distribution point 40 as distribution point 40 is added to 
communication system 20. Supervisor 56 provides each distribution point 40 with 
a logical address and a listing indicating to which additional distribution point 40 in 
communication with distribution point 40 information packets should be forwarded 
for each possible destination distribution point 40. The listing may be based on 
maintaining a minimum quality of service in the path through distribution point 
network 41 to the destination distribution point 40. Supervisor 56 periodically 
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assesses the performance of network 41 by sending test messages. Reports may also 
be generated by distribution points 40 attempting to communicate with target 
addresses. 

Supervisor 56 is shown in Figure 1 as a separate component 
5 individually connected to each distribution point 40. Alternatively, communication 
between supervisor 56 and distribution points 40 may be through radio links 46 and 
wireline links 48. Supervisor 56 may be one or more separate components of 
communication system 20, may be incorporated into one of distribution points 40, 
or may be distributed amongst multiple distribution points 40. 

10 In an embodiment of the present invention, a distribution point may 

be automatically added to or removed from distribution point network 41. When 
new distribution point 40 is first inserted into communication system 20, new 
distribution point 40 transmits a signature signal. Existing distribution points 40 
within range of new distribution point 40 receive the signal and report it to 

15 supervisor 56. Supervisor 56 then determines if new distribution point 40 will be 
added to network 41. If so, supervisor 56 assigns new distribution point 40 a 
routing address and informs network 41 as needed. Each existing distribution point 
40 in distribution point network 41 is provided with an indication as to which 
distribution point 40 in communication with existing distribution point 40 each 

20 information packet having a destination address specifying the new distribution point 
40 is to be forwarded. If a distribution point 40 is removed from network 41, 
remaining distribution points 41 report the absence of removed distribution point 40 
to supervisor 56. Supervisor 56 then informs network 41 as needed. 

In an embodiment of the present invention, each subscriber unit 26 
25 is autonomously registered with communication system 20 when subscriber unit 26 
first enters coverage area 24 within communication system 20. Each subscriber unit 
26 maintains registration as subscriber unit 26 moves from one coverage area 24 into 
another coverage area 24 within communication system 20 and is autonomously 
deregistered when subscriber unit 26 leaves communication system 20. To 
30 accomplish automatic registration and deregistration of subscriber units 26, each 
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access point 22 periodically reports the status of subscriber units 26 within any 
controlled coverage area 24 to supervisor 56 performing registration and 
authentication. Each access point 22 communicates with subscriber units 26 to 
determine status. When a subscriber unit 26 voluntarily enters or leaves coverage 

5 area 24, such as by powering up or down, subscriber unit 26 transmits a particular 
signal to access point 22. Information is also received from subscriber unit 26 in 
response to periodic queries from network 20. Access point 22 may determine the 
absence of subscriber unit 26 from coverage area 24 if no communication is received 
after a particular time interval. Algorithms for registering and deregistering 

10 subscriber units 26 may be based on various factors including quality of service, 
traffic, location, service type, network topology, and the like. 

Referring now to Figure 2, a block diagram of a distribution point 
according to an embodiment of the present invention is shown. Distribution point 
40 includes one or more front end communication interfaces 100, each front end 

15 interface communicating with one access point 22. In one configuration, access 
point 22 is packaged with distribution point 40. Front end interface 100 may 
provide a plug-in port for receiving access point 22. In another configuration, front 
end interface 100 connects to antenna 102 for establishing radio link 42 with access 
point 22. In a further configuration, front end interface 100 accepts wireline link 44 

20 connecting distribution point 40 with access point 22. Front end interface 100 
operates using a standard packet switching protocol such as, for example, ATM25. 
Each front end communication interface 100 passes information packets through 
common front end switch interface 104 operating under a packet protocol such as 
ATM, TCP/IP, 802.x, or the like. 

25 Distribution point 40 also includes back end communication interfaces 

106 for connecting distribution point 40 with additional distribution points 40, with 
supervisor 56, and, if distribution point 40 is a gateway 50, with telecommunication 
systems, private network systems, video distribution systems, the Internet, or the 
like. This may be typically referred to as back haul communication. In one 

30 configuration of the present invention, back end interface 106 connects to antenna 
108 for establishing radio link 46 with another distribution point 40. In another 
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configuration, back end interface 104 accepts wireline link 44 connecting distribution 
point 40 with another distribution point 40. In a preferred embodiment, back end 
interface 106 accepts modules 1 10 for interfacing through a variety of protocols and 
media such as ATM25, DS1, DS3, OC3, lOOOBase-T, 100Base-T, and the like. 
5 Each back end communication interface 106 passes information packets through 
common back end switch interface 112 operating under a packet protocol such as 
ATM, TCP/IP, or the like. In a preferred embodiment, distribution point 40 
dynamically allocates bandwidth when the information packet is forwarded to one 
of the additional distribution points 40 in communication with distribution point 40. 

10 Intelligent packet switch 114 received information packets through 

common front end switch interface 104 and common back end switch interface 112 
and routes the packets between front end interfaces 100 and back end communication 
interfaces 106. Switch 114 may be a packet switching device as is known in the art 
such as an ATM switch, an IP switch, a TDM switch, a switch working under the 

15 802.11 specification, or any other suitable alternative or combination having the 
required switching functionality. In an embodiment of the present invention, switch 
114 includes an ATM portion for routing voice, video and data, and an IP portion 
for real-time dynamic data routing and non-real time data routing as well as 
administration, management, and network topology control. 

20 In an embodiment of the present invention, distribution point 40 is 

enclosed in an environmentally sealed package. This permits distribution point 40 
to be mounted outside, such as on a pole or the side of a building. In keeping with 
the invention, however, distribution point 40 need not be outside so long as it can 
communicate with access points 22, additional distribution points 40, supervisor 56, 

25 and any other suitable network component. 

Referring now to Figure 3, a schematic drawing illustrating an 
implementation of a communication system according to an embodiment of the 
present invention is shown. This implementation provides an example including 
interfaces between communication system 20 and a variety of external 
30 communication systems 52. 
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Communication system 20 includes wireless service location registers 
(WSLRs) 200 providing common subscriber and service databases. Each WSLR 200 
communicates with at least one distribution point 40 and one additional 
communication system 52. Connections between WSLRs 200 and communication 

5 systems 52 are not shown in Figure 3 for clarity. Each WSLR 200 provisions 
requested services from additional communication system 52. For example, WSLR 
200 may provide centralized mobility and location management. Supervisor 56 
determines which WSLR 200 will provision services based on the distribution point 
40 through which subscriber unit 26 requesting services is currently communicating. 

10 A device that may serve as WSLR 200 is described in U.S. Patent No. 5,974,331 
titled "Method And System For Dynamically Assigning Features And Users To 
Wireline Interfaces," to Cook et al. Call agents may also function as WSLR-like 
devices to map or integrate additional communication systems with system 20. 

Communication system 20 may also include multi-service platform 
15 (MSP) 202. MSP 202 provides access to wireline telephone systems (PSTN). This 
may be accomplished through GR-303 compliant connection 204. Signaling point 
of interface (SPOI) 206 serves as the demarcation point between communication 
system 20 and external communication system 52. In the example shown, GR-303 
connection 204 connects wireline provider 208, serving wired customers 210, with 
20 communication system 20. MSP 202 may integrate both PSTN and IP networks as 
well as provide enhanced circuit/packet switch services. 

At least one gateway 212 supports MSP 202. Communication system 
20 may include, for example, voice-over-ATM (VoATM) to GR-303 gateways and 
voice over IP (VoIP) to GR-303 gateways. Gateway 212 serves as a protocol agent, 
25 converting information packets to a format acceptable to additional communication 
system 52. A determination as to which gateway 212 will process an information 
packet may be based on information contained within the information packet. 
Gateways 212 may be connected to MSP 202 by GR-303 compliant connection 214. 

Communication system 20 may also include gateway 216 connecting 
30 communication system 20 with external data network 52 such as the Internet or a 
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private data network interconnecting network users 218. Gateway 216 may, for 
example, convert between various packet-based standards such as H.323 and SIP. 

Communication system 20 may also include gateway 220 interfacing 
communication system 20 with external SS7 network 52 represented by signal 
5 transfer point (STP) 222. Gateway 220 communicates with STP 222 through ISUP 
compliant connection 224 which permits setting up and taking down trunk calls, 
calling party information services, call status, and any other suitable network 
function, by passing signaling information through SS7 network 52 to wireline 
provider 208 under the control of integrated services control point (ISCP) 226. 

10 Communication system 20 may also include unified message center 

(UMC) 228. Unified messages, also known as integrated messages, permit messages 
from a variety of sources such as telephone, email, fax, reports, compound 
documents, or any other suitable information or communication device, to be 
summarized and presented on a single medium, such as a personal computer. 

15 Messages may even be translated from one media type to another. UMC 228 
supports unified message applications within communication system 20. In an 
embodiment, UMC 228 communicates with wireline provider 208, permitting greater 
integration, flexibility and access to messages. 

Connection controller 230 controls access to gateways 50, 202, 212, 
20 216, 220, or any other suitable interface. For example, connection controller 230 
may manage voice over ATM to GR-303 access, voice over IP to GR-303 access, 
H.323/SIP to Internet remote access, SS7 to IP access, and the like. Connection 
controller 230 may also support information rate adaptation including open 
application processor interfaces and robust application development platforms. 

25 Referring now to Figures 4-6, drawings illustrating dynamic growth 

of a distribution point network according to an embodiment of the present invention 
are shown. An initial configuration for system 20 is shown in Figure 4. ATM/IP 
switch 300 is in communication with supervisor 56 through ATM virtual connection 
302. In this simple configuration, ATM/IP switch 300 may be functioning as access 
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point 22. ATM/IP switch 300 may obtain an IP address and an ATM address either 
manually or automatically. 

ATM/IP switch 300 automatically requests addresses by first 
broadcasting an ATM request in an IP packet over virtual connection (VC) 302. 

5 Supervisor 56 forwards this request to address server 304. Address server 304 
responds by allocating unique ATM end system addresses (AESA) 306 to the address 
assignment client in ATM/IP switch 300, which updates the ATM layer with new 
address 306. The address assignment client in ATM/IP switch 300 next requests 
from address server 304 an IP address, again using IP as the transport service over 

10 pre-existing ATM VC 302. Address server 304 forwards IP address 308 to ATM/IP 
switch 300. ATM/IP switch 300 then requests address pools for ATM and IP. 
Address server 304 responds by suppling AESA pool 310 and IP address pool 312. 
Pools of addresses 310, 312 are used by switch 300 when functioning as distribution 
point 40 in support of other distribution points 40 and access points 22. 

15 Referring now to Figure 5, a drawing illustrating the addition of a 

second access point is shown. Access point 22 has the capability to function as a 
distribution point 40. As new access points 22 are subtended from an existing access 
point 22, existing access point 22 becomes a distribution point 40. Each distribution 
point 40 continues to communicate with its initially connected distribution point 40 

20 and with other distribution points 40 as they are provisioned. As new links 46, 48 
between distribution points are created, distribution points 40 form peer relationships 
at both the ATM layer and the IP layer. Distribution points 40 are always peers at 
the IP and ATM layer, while access points 22 are clients of distribution points 40. 
As such, each ATM/IP switch 300 functions as a server when operating as 

25 distribution point 40 and as a client when operating as access point 22. 

When new ATM/IP switch 320 is instantiated as access point 22, it 
will automatically request an ATM address in an IP packet address assignment 
request. This IP packet will be sent in an ATM frame over radio link 46 to IP/ ATM 
switch 300 functioning as distribution point 40 using a pre-existing ATM VC. 
30 IP/ ATM switch 300 will allocate unique ATM address 322 from AESA pool 3 10 and 
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unique IP address 324 from IP address pool 312. ATM/IP switch 320 then sends 
a directed request to address server 304 and receives its own AESA pool 326 and IP 
address pool 328. 

Referring now to Figure 6, a hierarchical routing network is shown. 
5 Address server 304 assigns AESA pool 310 and IP address pool 312 as each 
ATM/IP switch 300 is added. By handling all requests for address pools 310, 312, 
address server 304 maintains a hierarchy of addresses for both ATM and IP layers. 
Address server 304 constructs routing tables for each ATM/IP switch 300 indicating 
to which directly connected ATM/IP switch 300 each incoming packet should be 

10 routed if the packet is not destined to subscriber unit 26 serviced by that ATM/IP 
switch 300. Thus, routing tables are cohesive, reflecting the view of communication 
system 20 seen by each ATM/IP switch 300. Address server 304 also constructs 
forward equivalency class (FEC) tables permitting ATM/IP switch 300 to route 
packages based on package contents. FECs can be seen as either the virtual path 

15 identifier (VPI) portion of the ATM VPI/VCI or as the entire VPI/VCI, and are 
enabled by the routing protocols at the IP and PNNI layers. 

New routing elements are dynamically added to a network of routing 
elements by establishing a connection between the new routing element and an 
existing routing element in the network of routing elements. At least one address 
20 is assigned to the new routing element, each assigned address coming from a pool 
of addresses maintained at the existing routing element. At least one pool of 
addresses is issued to the new routing element. The one or more pool of addresses 
permitting the new routing element to dynamically add yet another new routing 
element to the network of routing elements. 

25 In an embodiment of the present invention, a method of dynamically 

adding a routing element to a distributed communications includes establishing an 
ATM virtual connection with an existing distribution point already part of the 
communications network. An ATM end user address is requested from the existing 
distribution point. An ATM end user address is obtained from the existing 

30 distribution point, the ATM end user address allocated from a pool of ATM end user 
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addresses in the existing distribution point. An IP address is requested from the 
existing distribution point. An IP address is obtained from the existing distribution 
point, the IP address allocated from a pool of IP addresses in the existing distribution 
point. Preferably, a pool of ATM end user addresses is requested and received from 
5 an address server. A pool of IP addresses is requested and received from the address 
server. An ATM end user address from the pool of ATM end user addresses and 
an IP address from the pool of IP addresses may be assigned to a new routing 
element requesting to be added to the communications network. 

In an embodiment of the present invention, when a new distribution 
point is added to a network of distribution points, a connection is established 
between the new distribution point and at least one existing distribution point in the 
network of distribution points. A peer-to-peer relationship is formed at the OSI 
network layer between the new distribution point and the at least one existing 
distribution point. A peer-to-peer relationship is formed at the OSI data link layer 
between the new distribution point and the at least one existing distribution point. 

In an embodiment of the present invention, when an access point is 
added to a network of distribution points, a connection is established between the 
access point and at least one existing distribution point in the network of distribution 
points. A client-server relationship is formed at the OSI network layer between the 
20 access point client and the at least one existing distribution point server. A client- 
server relationship is formed at the OSI data link layer between the access point and 
the at least one existing distribution point server. 

In an embodiment of the present invention, when an ATM/IP switch 
300 is removed from communication system 20, all of the addresses 306, 308 and 
25 address pools 310, 312 associated with the removed switch 300 are released. 
Addresses 306, 308 and address pools 310, 312 may be instantiated at distribution 
point 40 which originally supplied removed switch 300 with addresses 306, 308, 
may be sent to supervisor 56, or may be split with addresses 306, 308 returning to 
distribution point 40 and address pools 310, 312 returning to supervisor 56. 
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When subscriber unit 26 first enters communication system 20, it is 
detected and serviced by access point 22 . Subscriber unit 26 is provided with one 
or more addresses, each address routable within at least the local hierarchy of 
ATM/IP switches 300. If subscriber unit 26 enters the range of a new access point 
5 22, new access point 22 sends out a flooding FEC routing update for the ATM 
address of subscriber unit 26. Previously servicing access point 22 removes 
subscriber unit 26 from its own FEC upon receiving the FEC update. :Any 
subsequent ATM packets received by previously servicing access point 22 are 
discarded. 

10 When subscriber unit 26 changes access points 22, the IP routing 

portion of system 20 moves the IP address of subscriber unit 26 from one FEC class 
to another. Any IP packets remain untouched, with only a label or equivalence 
changed. In one embodiment, the label is the VPI portion of the VC. In another 
embodiment, multiprotocol label switching (MPLS) is used to provide an additional 

15 label. In either case, the IP address and virtual connection identifier (VCI) are 
retained. 



Referring now to Figure 7, a schematic drawing illustrating a network 
of distribution points routing packets based on forwarding equivalency classes is 
shown. Each distribution point contains an FEC table 400 used to route packets 

20 received from and destined to subscriber units 26. Forwarding equivalency class 
table 400 contains one entry for each forwarding equivalency class 402. In the 
embodiment shown, there is one FEC 402 corresponding to each access point 22. 
When subscriber unit 26 is first detected by access point 22, the address for 
subscriber unit 22 is placed in the equivalency class 402 corresponding to the 

25 detecting access point 22. The address is typically an IP address. This may be done 
by broadcasting from detecting access point 22 or, preferably, is done by supervisor 
56 after supervisor 56 receives a message indicating subscriber unit 26 has been 
detected by access point 22. 

FEC table 400 in each distribution point 40 contains the next 
30 destination for each FEC 402. Typically, destinations are other distribution points 
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40, access points 22 serviced by distribution point 40, gateways 50, and other 
suitable points for routing, switching, servicing, distributing, and the like. When 
a packet destined for subscriber unit 26 is received by distribution point 40, 
distribution point 40 determines to which FEC subscriber unit 26 is assigned, 
5 determines to which destination the packets in that FEC are routed, and forwards the 
packet to the determined destination. 

Referring now to Figure 8, a schematic drawing illustrating 
forwarding equivalency class updating to track a moving subscriber unit is shown. 
If subscriber unit 26 is mobile or portable, as indicated by Ul in Figure 8, 

10 subscriber unit 26 may move out of coverage area 24 for one access point 22 and 
into coverage area 24 for a new access point 22. The address for subscriber unit 26 
is then moved from FEC 402 of original access point 22 into FEC 402 for new 
access point 22. In an embodiment, when original access point 22 is no longer in 
communication with subscriber unit 26, original access point 22 broadcasts a 

15 message to distribution points 40 and supervisor 56. If original access point 22 
subsequently receives any packets for subscriber unit 26, original access point 
forwards these packets back into distribution point network 41 . If the packets are 
sequentially indicated, such as IP packets, correct order will be established by 
subscriber unit 26 when packets are received. 

20 While embodiments of the invention have been illustrated and 

described, it is not intended that these embodiments illustrate and describe all 
possible forms of the invention. Rather, the words used in the specification are 
words of description rather than limitation, and it is understood that various changes 
may be made without departing from the spirit and scope of the invention. 
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WHAT IS CLAIMED IS: 

1 1 . A method of dynamically adding a new routing element to a 

2 network of routing elements comprising: 

3 establishing a connection between the new routing element and an 

4 existing routing element in the network of routing elements; 

5 assigning at least one address to the new routing element, each 

6 assigned address coming from a pool of addresses maintained at the existing routing 

7 element; and 

8 issuing at least one pool of addresses to the new routing element, the 

9 at least one pool of addresses permitting the new routing element to dynamically add 
10 yet another new routing element to the network of routing elements. 

1 2. A method of dynamically adding a new routing element as in 

2 claim 1 further comprising placing an address for each subscriber unit accessing the 

3 network of routing elements into at least one forward equivalency class based on the 

4 at least one assigned address. 

1 3 . A method of dynamically adding a new routing element as in 

2 claim 1 further comprising: 

3 determining that the new routing element is no longer connected to 

4 an existing routing element in the network of routing elements; 

5 releasing the at least one address assigned to the new routing element; 

6 and 

7 releasing the at least one pool of addresses issued to the new routing 

8 element 

1 4. A method of dynamically adding a new routing element as in 

2 claim 1 wherein establishing a connection between the new routing element and an 

3 existing routing element in the network of routing elements comprises establishing 

4 an ATM virtual connection with an existing distribution point already part of the 

5 communications network. 
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1 5. A method of dynamically adding a new routing element as in 

2 claim 1 wherein assigning at least one address to the new routing element comprises: 

3 requesting an ATM end user address from the existing distribution 

4 point; 

5 obtaining an ATM end user address from the existing distribution 

6 point, the ATM end user address allocated from a pool of ATM end user addresses 

7 in the existing distribution point; 

8 requesting an IP address from the existing distribution point; and 

9 obtaining an IP address from the existing distribution point, the IP 
10 address allocated from a pool of IP addresses in the existing distribution point. 

1 6. A method of dynamically adding a new routing element as in 

2 claim 5 wherein issuing at least one pool of addresses comprises: 

3 requesting a pool of ATM end user addresses from an address server; 

4 receiving a pool of ATM end user addresses from the address server; 

5 requesting a pool of IP addresses from the address server; and 

6 receiving a pool of IP addresses from the address server. 

1 7. A method of dynamically adding a new routing element as in 

2 claim 1 wherein the new routing element is a distribution point. 

1 8. A method of dynamically adding a new routing element as in 

2 claim 7 further comprising: 

3 forming a peer relationship with at least one existing distribution point 

4 at the data link layer for voice communication; and 

5 forming a peer relationship with the existing distribution point at the 

6 network layer for data communication. 

1 9. A method of dynamically adding a new routing element as in 

2 claim 1 wherein the new routing element is an access point interconnecting user 

3 communication devices with a distribution point. 
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1 10. A method of dynamically adding a new routing element as in 

2 claim 9 wherein the access point is a local radio access point serving a plurality of 

3 packet radios. 

1 1 1 . A method of dynamically adding a new routing element as in 

2 claim 9 further comprising: 

3 forming a client relationship at the data link layer with at least one 

4 existing distribution point, the existing distribution point functioning as a server for 

5 voice communication; and 

6 forming a client relationship at the network layer with the existing 

7 distribution point, the existing distribution point functioning as a server for data 

8 communication. 

1 12. A method of dynamically adding a new routing element as in 

2 claim 11 further comprising: 

3 adding at least one additional access point as a client; 

4 forming a peer relationship with communicating existing distribution 

5 points at the data link layer for voice communication; and 

6 forming a peer relationship with communicating existing distribution 

7 points at the network layer for data communication. 

1 13. A method of routing information packets in a distributed 

2 routing network comprising: 

3 determining a forwarding equivalency class for each subscriber unit 

4 accessing the network based on a point at which the subscriber unit accesses the 

5 network; 

6 updating the forwarding equivalency class for each subscriber unit if 

7 the subscriber unit accesses the network at a different point; and 

8 determining a next distribution element to forward an information 

9 packet based on the forwarding equivalency class for a destination subscriber unit. 

1 14. A method of routing information packets in a distributed 

2 routing network as in claim 13 further comprising distributing the forwarding 
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3 equivalency class for each subscriber unit to each distribution element in the 

4 distributed routing network. 

1 15. A method of routing information packets in a distributed 

2 routing network as in claim 13, the distributed routing network including a plurality 

3 of distribution points, each distribution point in direct communication with at least 

4 one additional distribution point in the network of distribution points, the distributed 

5 routing network also including an assigned access point for each subscriber unit 

6 accessing the communication system, the access point in communication with at least 

7 one distribution point in the network of distribution points, wherein determining a 

8 forwarding equivalency class is based on the access point. 

1 16. A method of routing information packets in a distributed 

2 routing network as in claim 15 further comprising distributing the equivalency class 

3 to each distribution point. 

1 17. A method of routing information packets as in claim 15 further 

2 comprising placing an IP address for each subscriber unit in the determined 

3 forwarding equivalency class. 

1 1 8 . A method of routing information packets as in claim 15 further 

2 comprising: 

3 determining that a subscriber unit should be assigned to a different 

4 access point; 

5 determining a new access point for the subscriber unit; 

6 determining a forwarding equivalency class for the subscriber unit 

7 based on the new access point; and 

8 notifying each distribution point in the network of distribution points 

9 of the forwarding equivalency class. 



1 19. A method of routing information packets as in claim 18 

2 wherein determining a new access point for the subscriber unit comprises 
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3 transmitting an indication for the new access point through the network of 

4 distribution points. 

1 20. A method of routing information packets as in claim 18 further 

2 comprising: 

3 determining that the subscriber unit is no longer accessing the network 

4 through an access point; and 

5 broadcasting an indication for the access point through the network 

6 of distribution points. 

1 21. A method of routing information packets as in claim 18 

2 wherein determining a forwarding equivalency class for the subscriber unit 

3 comprises: 

4 receiving a determination that the subscriber unit is accessing the 

5 network through an access point; and 

6 broadcasting the assignment of the forwarding equivalency class to 

7 each distribution point in the network of distribution points. 

1 22. A method of routing information packets as in claim 21 

2 wherein receiving is at a supervisor and broadcasting is from the supervisor. 

1 23. A communication system comprising: 

2 a network of distribution elements, each distribution element sending 

3 information packets to at least one additional distribution element, each distribution 

4 element maintaining at least one pool of addresses; and 

5 a new distribution element joining the network of distribution 

6 elements, the new distribution element operative to 

7 (a) establish a connection to an existing distribution element in 

8 the network of distribution elements; 

9 (b) receive at least one address from the existing distribution 

10 element, each received address coming from a pool of 

1 1 addresses maintained by the existing distribution element; and 
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12 (c) receive at least one pool of addresses, each pool of addresses 

13 permitting the new distribution element to dynamically add 

14 yet another new distribution element to the network of 

15 distribution elements. 

1 24. A communication system as in claim 23 wherein the received 

2 address comprises at least one from a set including an ATM address and an IP 

3 address. 

1 25 . A communication system as in claim 23 wherein the at least 

2 one received address comprises an ATM address and an IP address. 

1 26. A communication system as in claim 23 wherein the 

2 established connection is a wireline connection. 

1 27. A communication system as in claim 23 wherein the 

2 established connection is a wireless connection. 

1 28. A communication system as in claim 23 wherein the new 

2 distribution element further receives at least one forward equivalency class for 

3 routing information packets. 

1 29. A communication system as in claim 23 wherein the network 

2 of distribution elements is operative to: 

3 determine that the new distribution element is no longer connected to 

4 any existing distribution element in the network of distribution elements; 

5 recover the at least one address received from the existing distribution 

6 element; and 

7 recover the at least one pool of addresses. 

1 30. A communication system as in claim 23 wherein the network 

2 of distribution elements comprises: 
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3 a supervisor operative to supply a plurality of pools of ATM end user 

4 addresses and a plurality of pools of IP addresses; 

5 a plurality of access points, each access point operative to transmit 

6 packets to and receive packets form at least one subscriber unit; and 

7 a plurality of distribution points, each distribution point in 

8 communication with at least one other distribution point and with the supervisor, 

9 each access point in communication with at least one distribution point. 

1 31. A communication system as in claim 30 wherein the new 

2 distribution element is a new distribution point. 

1 32. A communication system as in claim 31 wherein the new 

2 distribution element is operative to 

3 (a) form a peer-to-peer relationship at the OSI network layer with 

4 the at least one existing distribution point, and 

5 (b) form a peer-to-peer relationship at the OSI data link layer 

6 with the at least one existing distribution point. 

1 33. A communication system as in claim 32 wherein the peer-to- 

2 peer relationship at the network layer governs exchanges of IP packets carrying data 

3 and control information and wherein the peer-to-peer relationship at the data link 

4 layer governs exchanges of ATM packets carrying voice information. 

1 34. A communication system as in claim 30 wherein the new 

2 distribution element is a new access point. 

1 35. A communication system as in claim 34 wherein the new 

2 distribution element is operative to 

3 (a) form a client-server relationship at the data link layer with the 

4 at least one existing distribution point, the new access point 

5 functioning as a client and the at least one existing distribution 

6 point functioning as a server, and 
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7 (b) form a client-server relationship at the network layer with the 

8 at least one existing distribution point, the new access point 

9 functioning as a client and the at least one existing distribution 
10 point functioning as a server. 

1 36. A communication system as in claim 35 wherein the client- 

2 server relationship at the network layer governs exchanges of IP packets carrying 

3 data and control information and wherein the client-server relationship at the data 

4 link layer governs exchanges of ATM packets carrying voice information. 

1 37. A communication system for distributing packetized 

2 information comprising: 

3 a plurality of access points, each access point operative to transmit 

4 packets to and receive packets from at least one subscriber unit accessing the 

5 communication system through the access point; and 

6 a plurality of distribution points, the access points and distribution 

7 points forming a distributed network, each distribution point in communication with 

8 at least one other distribution point, each access point in communication with at least 

9 one distribution point, each distribution point routing information packets received 

10 by the distribution point based on a forwarding equivalency class for a recipient 

11 subscriber unit specified in the information packet. 

1 38. A communication system as in claim 37 wherein access points 

2 are in wireless communication with distribution points. 

1 39. A communication system as in claim 37 wherein the distributed 

2 network is a wireless distributed network. 

1 40. A communication system as in claim 37 wherein at least one 

2 distribution point is in wireless communication with at least one other distribution 

3 point. 
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1 41. A communication system as in claim 37 wherein the 

2 forwarding equivalency class for a recipient user is determined based on the access 

3 point through which the recipient user is accessing the communication system. 

1 42. A communication system as in claim 37 wherein routing an 

2 information packet is based on an IP address within the information packet. 

1 43. A communication system as in claim 37 wherein each 

2 distribution point is further operative to change the forwarding equivalency class for 

3 a subscriber unit. 

1 44. A communication system as in claim 43 wherein the change 

2 in the forwarding equivalency class is based on a determination that the subscriber 

3 unit is accessing the communication system through a second access point different 

4 than a first access point through which the subscriber unit accessed the 

5 communication network. 

1 45. A communication system as in claim 44 wherein each 

2 distribution point is further operative to transmit an indication for the second access 

3 point. 

1 46. A communication system as in claim 44 wherein the subscriber 

2 unit is no longer accessing the communication system through the first access point, 

3 each distribution point further operative to transmit an indication of the first access 

4 point. 

1 47. A communication system as in claim 37 further comprising a 

2 supervisor operative to receive a determination that a subscriber unit is accessing the 

3 communication system through a second access point and to broadcast the 

4 assignment of a new forwarding equivalency class based on the second access point 

5 to the plurality of distribution points. 
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